近年来，神经网络已显示出巨大的增长，以解决许多问题。已经引入了各种类型的神经网络来处理不同类型的问题。但是，任何神经网络的主要目标是使用层层次结构将非线性可分离的输入数据转换为更线性可分离的抽象特征。这些层是线性和非线性函数的组合。最流行和常见的非线性层是激活功能（AFS），例如Logistic Sigmoid，Tanh，Relu，Elu，Swish和Mish。在本文中，在神经网络中为AFS提供了全面的概述和调查，以进行深度学习。涵盖了不同类别的AFS，例如Logistic Sigmoid和Tanh，基于RELU，基于ELU和基于学习的AFS。还指出了AFS的几种特征，例如输出范围，单调性和平滑度。在具有不同类型的数据的不同网络的18个最先进的AF中，还进行了性能比较。提出了AFS的见解，以使研究人员受益于进一步的研究和从业者在不同选择中进行选择。用于实验比较的代码发布于：\ url {https://github.com/shivram1987/activationfunctions}。

卷积神经网络（CNN）通常是使用基于随机梯度下降（SGD）优化技术训练的。现有的SGD优化器通常会遭受最小值和最低振荡的过度损失。在本文中，我们提出了一种新方法，以下内容称为Adainject，以将二阶时刻注入一阶时刻，以称为梯度下降优化器。具体而言，参数的短期更改被用作重量，以在更新规则中注入二阶时刻。 Adainject优化器控制参数更新，避免了最小值的过度换档，并减少了最小值接近的振荡。提出的方法本质上是通用的，可以与任何现有的SGD优化器集成。通过直观地解释了Anainject优化器的有效性以及一些玩具示例。我们还显示了拟议的基于注射的优化器的收敛性。此外，我们通过广泛的实验与最新的优化器（即Adaminject，diffgradinject，radaminject和Adabeliefinject在四个基准数据集中）一起描述了ADAIN方法的功效。实验中使用了不同的CNN模型。在CIFAR10数据集上使用resnext29模型，使用diffgradinject Optimizer观察到TOP-1分类错误率$ 16.54 \％$的最高提高。总体而言，我们通过提出的ADAIN方法观察到现有优化器的性能提高非常有希望。该代码可在：\ url {https://github.com/shivram1987/adainject}中获得。

随机梯度体面（SGD）是深神经网络成功背后的核心技术之一。梯度提供有关功能具有最陡变化率的方向的信息。基本SGD的主要问题是通过梯度行为而对所有参数的相等大小的步骤进行更改。因此，深度网络优化的有效方式是为每个参数进行自适应步骤尺寸。最近，已经进行了几次尝试，以改善梯度下降方法，例如Adagrad，Adadelta，RMSProp和Adam。这些方法依赖于平方过去梯度的指数移动平均线的平方根。因此，这些方法不利用梯度的局部变化。在本文中，基于当前和立即梯度（即，差异）之间的差异提出了一种新颖的优化器。在所提出的差异优化技术中，以这样的方式调整步长，使得它应该具有更大的梯度改变参数的较大步长，以及用于较低梯度改变参数的较低步长。收敛分析是使用在线学习框架的遗憾方法完成。在本文中进行严格的分析超过三种合成复合的非凸功能。图像分类实验也在CiFar10和CiFAR100数据集上进行，以观察漫反射的性能，相对于最先进的优化器，例如SGDM，Adagrad，Adadelta，RMSProp，Amsgrad和Adam。基于基于单元（Reset）的基于卷积神经网络（CNN）架构用于实验中。实验表明，Diffgrad优于其他优化器。此外，我们表明差异对使用不同的激活功能训练CNN的均匀良好。源代码在https://github.com/shivram1987/diffgrad公开使用。

Machine Translation (MT) system generally aims at automatic representation of source language into target language retaining the originality of context using various Natural Language Processing (NLP) techniques. Among various NLP methods, Statistical Machine Translation(SMT). SMT uses probabilistic and statistical techniques to analyze information and conversion. This paper canvasses about the development of bilingual SMT models for translating English to fifteen low-resource Indian Languages (ILs) and vice versa. At the outset, all 15 languages are briefed with a short description related to our experimental need. Further, a detailed analysis of Samanantar and OPUS dataset for model building, along with standard benchmark dataset (Flores-200) for fine-tuning and testing, is done as a part of our experiment. Different preprocessing approaches are proposed in this paper to handle the noise of the dataset. To create the system, MOSES open-source SMT toolkit is explored. Distance reordering is utilized with the aim to understand the rules of grammar and context-dependent adjustments through a phrase reordering categorization framework. In our experiment, the quality of the translation is evaluated using standard metrics such as BLEU, METEOR, and RIBES

Unpaired exemplar-based image-to-image (UEI2I) translation aims to translate a source image to a target image domain with the style of a target image exemplar, without ground-truth input-translation pairs. Existing UEI2I methods represent style using either a global, image-level feature vector, or one vector per object instance/class but requiring knowledge of the scene semantics. Here, by contrast, we propose to represent style as a dense feature map, allowing for a finer-grained transfer to the source image without requiring any external semantic information. We then rely on perceptual and adversarial losses to disentangle our dense style and content representations, and exploit unsupervised cross-domain semantic correspondences to warp the exemplar style to the source content. We demonstrate the effectiveness of our method on two datasets using standard metrics together with a new localized style metric measuring style similarity in a class-wise manner. Our results evidence that the translations produced by our approach are more diverse and closer to the exemplars than those of the state-of-the-art methods while nonetheless preserving the source content.

We introduce an end-to-end computational framework that enables hyperparameter optimization with the DeepHyper library, accelerated training, and interpretable AI inference with a suite of state-of-the-art AI models, including CGCNN, PhysNet, SchNet, MPNN, MPNN-transformer, and TorchMD-Net. We use these AI models and the benchmark QM9, hMOF, and MD17 datasets to showcase the prediction of user-specified materials properties in modern computing environments, and to demonstrate translational applications for the modeling of small molecules, crystals and metal organic frameworks with a unified, stand-alone framework. We deployed and tested this framework in the ThetaGPU supercomputer at the Argonne Leadership Computing Facility, and the Delta supercomputer at the National Center for Supercomputing Applications to provide researchers with modern tools to conduct accelerated AI-driven discovery in leadership class computing environments.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Nostradamus, inspired by the French astrologer and reputed seer, is a detailed study exploring relations between environmental factors and changes in the stock market. In this paper, we analyze associative correlation and causation between environmental elements and stock prices based on the US financial market, global climate trends, and daily weather records to demonstrate significant relationships between climate and stock price fluctuation. Our analysis covers short and long-term rises and dips in company stock performances. Lastly, we take four natural disasters as a case study to observe their effect on the emotional state of people and their influence on the stock market.

We consider private federated learning (FL), where a server aggregates differentially private gradient updates from a large number of clients in order to train a machine learning model. The main challenge is balancing privacy with both classification accuracy of the learned model as well as the amount of communication between the clients and server. In this work, we build on a recently proposed method for communication-efficient private FL -- the MVU mechanism -- by introducing a new interpolation mechanism that can accommodate a more efficient privacy analysis. The result is the new Interpolated MVU mechanism that provides SOTA results on communication-efficient private FL on a variety of datasets.

Many modern computer vision algorithms suffer from two major bottlenecks: scarcity of data and learning new tasks incrementally. While training the model with new batches of data the model looses it's ability to classify the previous data judiciously which is termed as catastrophic forgetting. Conventional methods have tried to mitigate catastrophic forgetting of the previously learned data while the training at the current session has been compromised. The state-of-the-art generative replay based approaches use complicated structures such as generative adversarial network (GAN) to deal with catastrophic forgetting. Additionally, training a GAN with few samples may lead to instability. In this work, we present a novel method to deal with these two major hurdles. Our method identifies a better embedding space with an improved contrasting loss to make classification more robust. Moreover, our approach is able to retain previously acquired knowledge in the embedding space even when trained with new classes. We update previous session class prototypes while training in such a way that it is able to represent the true class mean. This is of prime importance as our classification rule is based on the nearest class mean classification strategy. We have demonstrated our results by showing that the embedding space remains intact after training the model with new classes. We showed that our method preformed better than the existing state-of-the-art algorithms in terms of accuracy across different sessions.